Aiming at the characteristics of rail surface images such as uneven illumination, limited discernible features, low contrast and changeable reflection characteristics, a background differential rail surface defect detection method based on defect proportion limitation was proposed. The method mainly includes five steps:pre-processing of rail surface images, background modeling and difference, defect proportion limitation filtering, maximum entropy threshold segmentation of defect proportion limitation and connected area labeling. Firstly, the column grayscale mean and median of the rail surface image were combined to perform the rapid background modeling, and the difference operation was carried out to the pre-processed image and the background image. Secondly, the feature with low defect proportion in the rail surface image was used to truncate the upper threshold limit of the defect proportion in order to enhance the contrast of the difference image. Thirdly, the maximum entropy threshold segmentation was improved by using this feature, the global variable weighting of the target entropy was carried out by using the adaptive weighting factor, and an appropriate threshold was selected to maximize the entropy value, so as to reduce the interference of noises such as shadow and rust while retaining the real defects. Finally, the connected area labeling method was used to perform the statistics of the defect areas in the segmented binary image, and the area with defect area lower than the rail damage standard was determined as the noise and removed, so as to realize the rail surface defect detection. Simulation results show that the new method can detect rail surface defects well, and its results have the recall rate, precision rate and weighted harmonic mean of 94.19%, 88.34% and 92.96% respectively, and the average mis-classification error of 0.006 4, so that the method has certain practical value.